Television 


Prof. Mohamed Nasr 


Any display system that 


Has digital processing of one or more of its inputs 
Includes the function of showing video 


Divide into 4 segments 

- Monitor/TV 

• Digital monitor with video function - 15" XGA LCD Monitor/TV 

- TV/Monitor 

• HD-Ready TV / EDTV / SDTV / 100 Hz TV - digital displays often 
include monitor as additional function 

MPEG TV 


• Integrated HDTV (USA); iDTV [Integrated Digital TV] (Europe); BS 
Digital (Japan) 

- Smart (IP) TV 

• Internet connectivity, built-in hard-drive (PVR), interactivity etc 


High Definition and Digital TV 


Digital and HD Available Today! 

Soon (one year from now) Analog Broadcasts 
Cease 

What Does This Mean To Us? 

- Why Do You Care? 

- What Do I Need? 

- What Will Happen to My Captions? 



What's Changing 


• Originally TVs Only Received "Over the Air" Analog 

• Then came Cable 

• And Satellite 

• Then came VCRs 

• And DVDs 

• And TiVos and DVRs 

• And the Internet 

• And Computers 

• And Digital and Video Cameras 

• And High Definition and Digital TV 

• Then (Feb 17, 2009) Analog Broadcasts Cease 



Why is it Changing? 


No snow, no static, no ghosts 
Higher resolution images, and Sound 
Additional channels 

Frees up Bandwidth for Other Important Needs 
More Options 

- Improved captions (but not without pain) 

- Interactively 

- TV to your Cell Phone or PDA? (maybe later) 
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Analog vs. Digital 


Analog 

- An analog signal is one that 
continuously varies and the 
strength of the signal carries the 
information. 



Digital 

- A digital signal is one that contains 
coded information that carries the 
information. 
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Analog vs. Digital 


Analog 

- Records 

- Film (movies, and photos) 

- Most VCRs 

- Most TVs (except the new ones) 

- Waves (sound, radio, light, water... nobody's sure about light) 

- All hearing aids, until about 10 years ago 

Digital 

- DVDs 

- CDs 

- Digital Photos 

- Computers 

- The Internet 

- New TVs 

- Satellite Radios 

- All CIs and Almost all Hearing Aids 



Analog TV 



La£l I lie 


visible 


Invisible 
Vertical 

Blanking Electron bairn 
Interval 


Horizontal retrace 


Vertical retrace 


Captions are 
Contained as 
An analog 
Signal in the 
Vertical Blanking 
Interval 



Digital TV 


Signal is just a bunch of bits 

Define color and intensity of 
each point on the screen 

Bit stream is heavily 
compressed 

Captions are also digital 
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And That's the News, Tonight 
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Need to Understand 


• What Will Work With What? 

• What Will I Need? 

• How to Connect your TV 

— Primary Source 

- DVDs and Tapes 

• Controlling Captioning Options 

- Where Decoded 

- Appearance 

- Turning on/off 

- CC vs. Subtitles 



What Will Work With What? 


Analog TV Will Continue To Work with 

Cable Set Top Box (STB) 

Satellite STB 

Digital/Analog Converter (receive digital broadcasts via antenna) 
Antenna will also receive analog broadcasts (until next year) 
Current VCRs and DVDs will work (with some limitations) 

Digital HD TVs Will Work with 

Cable (with or without STB) 

Satellite STB 

Antenna (if nearby digital stations exist) 

Current and Future VCRs and DVDs (with some limitations) 



For a Digital TV 


• Monitor vs. TV (some older models don't have a tuner) 

• Plasma vs. LCD 

• Projection vs. non-projection and DLP 

• Refresh Rates 

• Contrast Ratios 

• But you WON'T need a Digital/Analog Converter 

• Cable, or Satellite, or Antenna (if nearby digital broadcasts) 

• STB for Satellite and (maybe) for Cable 

• Connectors 

• Optional: DVDs, VCRs, DVRs 



For an Analog TV 


Must have at least one: 

— Cable (with STB) 

— Satellite (with STB) 

— Digital/Analog Converter and Antenna (and nearby digital stations) 
TV will still work from VCR or DVD even without those 
Picture may be quite a bit better, but it won't be HD 

— Depends on your TV 

— Depends on your connectors (S-Video is better than composite) 

Current and Future VCRs, DVDs and DVRs (with some 
limitations) 



Digital/Analog Converters 


What?: A Box that converts a digital signal to analog 


Why?: So analog TVs can display digital broadcasts 


Do I Really Need a D/A Converter? 

- No: If you have Cable or Satellite or if all your TVs are digital (with tuners) 

- Yes: to receive Over the Air on an Analog TV after 17 Feb 2009 


Reasons You May Want to Get a D/A Converter for Your Analog TVs 

- Pictures on some analog TVs may actually look better than they do now. 

- Captions may look better on some programs 

- Using analog TVs as extra TVs without cable/satellite 

- Using analog TVs for emergencies or cable/satellite outages 

- Some local multi-casts may not be on cable/satellite 



Connector Overview 


Coaxial Cable 
Composite *» 





From Cable, Satellite or Antenna to set top box or TV 


i 


Yellow is video + Red and White for Audio 


S-Video 

Component 




Carries Video with improved Quality 


Red, Blue, Green (RGB) is Video, + Red and White for Audio 
Digital quality and does captions 



Newest, Best all digital Video and audio, 
but doesn't do encoded captions 


System Interfaces 


RF - NTSC 
RF - ATSC 




Baseband analog NTSC 

- Composite (CVBS) 

- S-Video (Y/C) 

- Component (YUV) 

Analog HD component (YPbPr) 

Analog PC graphics (VGA) 

Digital PC graphics (DVI-HDCP) 

Digital HD 

- DVI-HDCP [High Definition Content Protection] from PC space used by STBs and current 
generation of HD-Ready TV 

HDMI - New CE version of DVI adds audio, video formats, control functions 








Connector Overview 


Antenna 


Old connector for over the air; often now have Coax for Antenna 


Optical 



No electronic interference, uses light (Audio Only) 


Firewire 



For Computers and Video Cams but doesn't do captions 


DVI 

VGA 

USB 



*> F) *| DVI-D and DVI-I: Good all digital Video, but doesn't do 
captions 

Use your TV as a computer monitor 



Commonly used to plug in digital cameras and see videos or slide 
shows 


Flash 


Sui Disk 31 

512mb 



Some TVs may have slots for these for viewing photos or video 
clips 




Connecting To Analog TV 



Old or Small TVs without 
Caption Decoder Chip can 
Use an external Caption 
Decoder Box. 



Connecting To Analog TV 



VCR 



Connecting To Analog TV 
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Connecting To Analog TV 




Connecting To Digital TV 



Digital TV with tuner (not just a 
"monitor" or "HD Ready" TV) 
can receive directly from 
antenna. 

Digital TV with QAM input can 
receive limited cable options 
without a Set Top Box 



Connecting To Digital Monitor 



Digital TVs without a Tuner 
(sometimes called "HD Ready" 
or a "Monitor") must have a 
digital tuner to receive digital 
signals. 

Indoor antenna may be enough 
in areas near a lot of digital 
stations. 


High Definition Multimedia 
Interface (HDMI) 



HDMI is DVI plus 


- Audio 


- Support for YCbCr video 

- CE control bus 

- Additional control and configuration capabilities 

- Small CE-friendly connector 

HDMI enables device communication 

- To source 

• Supported video and audio formats 

- To display 



Video and audio stream information 





The Modern "HD Ready" TV Set 



• The tuner extracts one TV channel at a time from many, then downconverts 
and demodulates the signal to "baseband" 

• The video (or "color") decoder separates the colors from the "composite" 
(CVBS) signal 

• The deinterlacer and scaler converts the format of the picture to match that 
of the display type (optional for CRT TVs) 

• The display electronics converts the signal format to match that of the 
display type: e.g. analog for CRT, LVDS for LCD panel 









Functional System Architecture for 

MPEG TV 
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Color Television Systems 


Color TV systems developed in '50s (NTSC) and '60s (PAL) 

Backward compatibility with monochrome TVs more important 
than color quality! 

- Basic parameters of signal (carrier frequencies, bandwidths, 
modulation format, etc.) had to remain unchanged 


NTSC and PAL systems added chrominance (color) information to 
luminance (brightness) signal in a manner transparent to 
monochrome TVs 



NTSC Fundamentals 


Approved in US by FCC in 1953 as color system compatible with existing 
525 line, 60 fields/sec, 2:1 interlace monochrome system 
Color added to existing luminance structure by interleaving luma and 
chroma in frequency domain 
Basic properties 

525 lines/frame 

- 2:1 interlace 2 fields/frame with 262.5 lines/field 
Field rate 59.94 Hz 

Line frequency (f h ) = 15.734 KHz 

- Chroma subcarrier frequency (f sc ) = 3.58MHz = 227.5 f h = 119437.5 f v 

• chosen so that consecutive lines and frames have opposite (180°) phase 

Luma: Y = 0.299R' + 0.587 G' + 0.114 B', where R', G', B' are gamma-corrected R, G, B 
Chroma: I (In-phase) and Q (Quadrature) used instead of color difference signals U, V 

• U = 0.492 (B'-Y), V = 0.877 (R'-Y) 

• I = V cos33° - U sin33°, Q = V sin33° + U cos33° 

- Composite = Y + Q sin(wt) + I cos(wt) + sync + blanking + color burst, 

where w = 2 pi f sc 


Monochrome TV Signals (NTSC) 


Video Audio 

Carrier Sub-Carrier 



(Amplitude Modulation/Vestigial Sideband) modulated onto the video 


carrier 

• The sound signal is FM modulated onto the Audio Sub-Carrier located 4.5 
MHz from the video carrier 


Spectrum of Monochrome TV Signal (NTSC) 


Video 

Carrier 




= Frame freq. = Line freq. 

(30 Hz) (15.734 kHz) 



Spectrum of the video extends from just below the video carrier 
frequency to just below the sound carrier 

Repetitive nature of the signal from line to line and frame to frame 
results in a "picket-fence", or comb, spectrum 


Color in Video 


In PC space, R+G+B signals generate color images 

In video space, color signals developed for backward compatibility with 
monochrome TVs 

• Image brightness represented by luma signal (Y), equivalent of 
monochrome TV signal 

• Color added with "color difference" signals: Cb and Cr 

• Matrix equation translates color spaces 

• Y (luma) = 0.299R' + 0.587G' + 0.114B' 

• Cb (blue chroma) = 0.492(B'-Y) 

• Cr (red chroma) = 0.877(R'-Y) 


Principles of NTSC Color System 


Takes advantage of spectral nature of luminance signal 

Recognizes human eye is less sensitive to color changes than luma 
changes 

Low bandwidth chrominance information is modulated onto a Color 
Sub-Carrier and added to the luma signal 

The chroma signal has a picket-fence spectrum 

- sub-carrier frequency very carefully chosen so of the chroma 
signal pickets are interlaced between those of the luma signal 

- f sc = 227.5 x f H = 3.579545 MHz 



Why a weird number like 59.94 Hz? 


Early TV systems used local power line frequency as the field rate 
reference 

Europe used 50 Hz, the USA used 60 Hz 

With the introduction of color, audio subcarrier frequency required 
integer relationship to color subcarrier to prevent interference 

- Nearest value to the original 4.500 MHz was 4.5045 MHz, 
too large a difference for backward compatibility 

Reducing field rate from 60 to >9.94 Hz, allowed integer value of 
4.49999 MHz possible for audio subcarrier 

- This is close enough, solving the problem 


The Result 


Chroma Luma 



The chroma components can be mostly separated from the luma with a 
comb filter 

Note the mixing of lower-level luma and chroma components, resulting 
residual cross-luma and cross-color artifacts 


Implementation of NTSC Color 


Gamma correction applied to adjust for CRT non-linearity of Component 
color signals (R 1 , G' and B') are converted to luma and chroma-difference 
signals with a matrix circuit: 

Cb and Cr are lowpass filtered, then quadrature modulated (QAM) onto 
the chroma sub-carrier 

- Signal Amplitude represents the color saturation of video 

- Phase represents the hue 

- Chroma levels chosen such that peak level of composite signal does 
not exceed 100 IRE with 75% color bars 




Spectrum of the NTSC Color Signal 


Video Color Audio 



• Full chroma signal bandwidth, ±1.3 MHz around sub-carrier, too wide for transmission within 
channel allocation 

• Usually, both Cb and Cr bandwidths are reduced to 600 kHz 

• Reduces cross-color and cross-luma in TV 

• Alternatively, compliant to true NTSC specification: 

- Cb component (only) can be band-limited to 600 kHz 

- Phase of sub-carrier rotated by 33°, puts flesh tones at around 0° 

— Results in asymmetrical signal (shown) 

- The rotation aligns flesh tones to the I axis and is transparent to demodulator since color-burst is rotated by same 
amount 


The NTSC Color Video signal 
(EIA 75% color bar signal) 



White 

level 


NTSC Color Video Signal 

EIA 75% Color Bar Signal 
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Blank 

level 


Sync 

level 



PAL Fundamentals 


European standard with many flavors - broadcasting begun in 1967 in 
Germany and UK. 

Similar in concept to NTSC, except that line and field timings are different 
and the phase of the V (chroma) component is reversed every line to 
allow color phase errors to be averaged out 

Basic properties (except for PAL-M which has NTSC like rates) 

625 lines/frame 

- 2:1 interlace ^ 2 fields/frame with 312.5 lines/field 
Field rate 50 Hz 

Line frequency (f h ) = 15.625 KHz 

- Chroma subcarrier frequency (f sc ) = 4.43MHz = (1135/4 + 1/625) f h 

* consecutive lines and frames have 90° phase shift, so 2 lines or 2 frames required for opposite phase 
Luma: Y = 0.299R' + 0.587 G' + 0.114 B', where R', G', B' are gamma-corrected R, G, B 

- Chroma: Usual color difference signals U, V 

• U = 0.492 (B'-Y), V = 0.877 (R'-Y) 

- Composite = Y + U sin(wt) +/- V cos(wt) + sync + blanking + color burst, 

where w = 2 pi f sc 


SECAM Fundamentals 


Developed in France - broadcasting begun in 1967 

Basic timing is identical to PAL but chroma is handled differently from 

NTSC/PAL 

Only one chroma component per line 
FM modulation is used to transmit chroma 
Basic properties 

- 625 lines/frame 

- 2:1 interlace 2 fields/frame with 312.5 lines/field 

- Field rate 50 Hz 

- Line frequency (f h ) = 15.625 KHz 

Luma: Y = 0.299R' + 0.587 G' + 0.114 B', where R', G', B' are gamma- 
corrected R, G, B 

Chroma: Scaled color difference signals U, V 

• Db = 1.505 (B'-Y), Dr = -1.902 (R'-Y) 

• only one chroma component per line, alternating between Dr, Db 

• separate subcarriers for Dr, Db 


Composite Video: NTSC / PAL / SECAM 


* Long-time world television standards 

• Basic properties 

- Analog interlaced scanning 

* 3D (H,V,T) information expressed as a ID (temporal) raster scanned signal 

* Each picture (frame) displayed as 2 interleaved fields - odd + even 

Luminance (Y) and Chrominance (R-Y, B-Y), sync, blanking, and color reference 
information all combined into one "composite" signal 
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Luma/Chroma Separation 


Many approaches trading off complexity, cost and performance 
Basic approaches (historical) 

* Low pass luma / high pass chroma 

* Notch luma / bandpass chroma 

Advanced approaches (commonly used in most systems today) 

* 2D passive line comb filter 

* 2D adaptive line comb filter 

* 3D (spatio-temporal) comb filter 

Decoding artifacts 
Loss of resolution 
Dot-crawl 
Cross-color 

Good decoding requires some black magic (art) because luma and 
chroma spectrums overlap in real motion video 


S-Video Signals 


S-Video was developed in conjunction with the 
S-VHS VCR standard, where the luma and chroma 
signals are kept separate after initial Y/C separation 

-Keeping the signals separate, i.e. never adding the luma and 
chroma back together, eliminates the NTSC artifacts 

-Since video sources are generally composite (NTSC), the full 
benefit is not realized 

- Keeping the signals separate after playback with the VCR does 
help, especially because of the timing jitter 


Luma and Chroma Signal Separation 

(Y/C Separation) 


The chroma signal (C) is separated from the composite signal by filtering 
- Adaptive comb filtering is required for high quality 

• Low cost TVs use a bandpass filter, resulting in incomplete 
separation and bad cross luma and chroma artifacts 

• Non-adaptive comb filters introduce problems at edges 
The luma signal (Y) may be derived by subtracting 

the chroma from the composite signal 

— Only works well if the chroma was separated well 

— Low cost TVs use a bandstop filter to eliminate the chroma, resulting 
in poor luma bandwidth 



NTSC Color Video signal after Y/C Separation 

(EIA 75% color bar signal) 
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The NTSC Color Video signal after chroma demodulation 

(EIA 75% color bar signal) 




Chroma Demodulation 


The Cb and Cr color difference signals are recovered by 
coherent demodulation of the QAM chroma signal 


An absolute phase reference is provided to facilitate the 
process 

- A color burst - 9 cycles of unmodulated color sub-carrier - 
is added between the horizontal sync pulses and the start 
of the active video (the "backporch") 



Notch/LPF versus Comb Filtering 


* Comb filtering allows full bandwidth decoding 




Comb Filtering (cont.) 


Use 1 or more lines for each delay element, e.g., for NTSC, D = 1 line = 910 
Apply "cos" version with positive coefficients to extract chroma or "sin" 
version with negative center coefficient to extract luma 
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Y/C separation using simple 2D passive comb 


HDTV Technical Overview 


Video: 

- MPEG2 Main Profile @ High Level (MP@HL) 

- 18 formats: 6 HD, 12 SD 

Audio: 

- Dolby AC-3 
Transport: 

- Subset of MPEG2 

- Fixed length 188-byte packets 

RF/Transmission: 

- Terrestrial: 

• 8-VSB (Vestigial Side Band) with Trellis coding 

• effective payload of ~19.3 Mb/s (18.9 Mb/s used for video) 

- Cable: 

• Uses QAM instead of VSB 

• effective payload of ~38.6 Mb/s 


ATSC Formats 


18 formats: 6 HD, 12 SD 

- 720 vertical lines and above considered High Definition 

- Choice of supported formats left voluntary due to disagreement between 
broadcasters and computer industry 

• Computer industry led by Microsoft wanted exclusion of interlace and initially 
use of only those formats which leave bandwidth for data services - "HDO" 
subset 

- Different picture rates depending on motion content of application 

• 24 frames/sec for film 

• 30 frames/sec for news and live coverage 

• 60 fields/sec, 60 frames/sec for sports and other fast action content 

1920 x 1080 @ 60 frames/sec not included because it requires ~100:1 compression to 
fit in 19.3 Mb/s terrestrial channel, which cannot be done at high quality with MPEG2 



HDTV/DTV System Layers 

layered system with header/descriptors 


Picture 

Layer 


996 Mb/s 

1920 x 1080 @601 



Multiple Picture Formats 
and Frame Rates 
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Chroma and Luma 
DCT Coefficients 


MPEG-2 video 
and Dolby AC-3 
compression 
syntax 
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Flexible delivery of data 
and future extensibility 


MPEG-2 packets 


Transmission 

Layer 


19.3 Mb/s 
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Aspect Ratios 


• Two options: 16:9 and 4:3 

• 4:3 standard aspect ratio for US TV and computer monitors 

• HD formats are 16:9 

better match with cinema aspect ratio 
better match for aspect ratio of human visual system 
better for some text/graphics tasks 
• allows side-by-side viewing of 2 pages 


600 


800 




4:3 aspect ratio 


16:9 aspect ratio 


Additive White Gaussian Noise 


Ubiquitous in any electronics systems where analog is present 
- Central Limit Theorem explains the underlying cause 

Noise can be dramatically reduced by motion-adaptive recursive filtering 
("3D NR") 


Basic equation: 

Yi = X + Zi 

where Zi = measurement at time I, X = original data 

Wi = noise at time i = Gaussian white noise with zero mean 

MMSE estimate for N measurements = Z(Yi)/N 

Compute Average over same pixel location in each frame 
Noise averages to zero over a period of time 

Since averaging pixels that are in motion produces tails, we need reduce or 
stop averaging when there is motion 


AWGN - Example 



Original 



With Gaussian noise 




Impulse Noise Reduction 

* Use nonlinear spatial filtering to remove impulsive noise without 
reducing resolution 



With impulse noise 





Digital (MPEG) Noise 


Block Noise 

Tiling effect caused by having different DC coefficients for 
neighboring 8x8 blocks of pixels 

Mosquito Noise 

Ringing around sharp edges caused by removal of high-frequency 
coefficients 

Noise reduction is achieved by using adaptive filtering 

Different choice of filters across block boundaries versus within 
blocks 


Deinterlacing (Line Doubting) 


Conversion of interlaced (alternate line) fields into progressive (every line) 
frames 

Required to present interlaced TV material on progressive display 

Odd Even 


CRT-TV uses Interlaced scanning, with odd 
lines first followed by even lines 




PC Monitor and all digital displays are 
Progressive - scanning all lines in 
consecutive order 



Vertical-Temporal Progression 
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Interlacing and Deinterlacing 
Artifacts 


• Interlacing artifacts 

- Twitter 

- Wide-area flicker 

- Temporal aliasing 
Line Crawl 

* Deinterlacing artifacts 

- Feathering/ghosting 

- Jaggies/stepping 

- Twitter 

Loss of vertical detail 

- Motion judder 

- Motion blur 

- Specialized artifacts 


Methods of Deinterlacing 


* Spatial interpolation ("Bob") 

* Temporal interpolation ("Weave") 

* Spatio-temporal interpolation 

* Median filtering 

* Motion-adaptive interpolation 

* Motion-compensated interpolation 

* Inverse 3-2 and 2-2 pulldown (for film) 

* Other (statistical estimation, model-based etc) 


Film vs. Video 


Nature of content is the most important factor 
Fundamentally two types - Progressive and Interlaced 

Progressive content is content that was originally acquired in progressive form 
but converted to fit into an interlaced standard 

Most common form of such content is film - 24 frames/sec or 30 
frames/sec 

Other forms include computer graphics/animation 
Film-to-video (Telecine) process is used to convert film to the desired interlaced 
video format 

24 frames/sec 50 fields/sec PAL by running film at 25 fps and 
doing "2:2 pulldown" 

24 frames/sec -> 60 fields/sec NTSC by doing "3:2 pulldown" 


Film-to-Video Transfer (NTSC) 
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* Conversion of 24 frames/sec into 60 fields/sec: 4 movie frames mapped to 5 
video frames 

* In this process, one movie frame is mapped into 3 video fields, the next into 
2, etc... 

• Referred to as "3:2 Pulldown" 

• Similar process used to convert 25 frames/sec to 50 fields/sec and 30 
frames/sec to 60 fields/sec ("2:2 pulldown") 


De-Interlacing of Film-Originated 
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De-Interlacing of Film-Originated 

Material 


Without Film Mode 
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Odd and even lines are in different places when there 
is motion 


Video Deinterlacing Artifact 
Feathering 


• Feathering - caused by improper handling of motion 



Moving Edges in Video 


• Hardest problem in de-interlacing because odd and even lines are in different 
places 

• Combining odd and even lines causes feathering 

• Using spatial interpolation causes jaggies/staircasing 


Angled Line Line Doubled using 

Vertical Interpolation 



Video Deinterlacing Artifact - 
J aggies / Staircasing 


• Jaggies/staircasing 

Caused by vertical interpolation across lines in same field 




Optima / Deinterlacing 


Content-adaptive 

Film vs. Video - detect film and use inverse 3-2 (NTSC) or 
inverse 2-2 (PAL) pulldown 

Bad edit detection/compensation - need to detect and 
compensate for incorrect cadence caused by editing 
Motion-adaptive 

Detect amount of motion and use appropriate mix of spatial 
and temporal processing 

* Highest resolution for still areas with no motion artifacts in moving 
areas 

Edge-adaptive 

Interpolate along edge to get smoothest/most natural image 


Film-Mode: Inverse Pulldown 


From movie From movie From movie From movie 



• Odd and even fields generated from the same original movie frame 
can be combined with no motion artifacts 

• "3:2 Pulldown" sequence detection is necessary 

• Done by analysis of motion content 


Bad Edit Detection and Correction 


• There are 25 potential edit breaks 

- 2 Good edits 

23 distinct disruptions of the film chain that cause visual bad edits 

* Sequence has to be continuously monitored 

From movie Edit From movie From movie 



Film to video transitions - commercial insertion or news flashes 
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Motion-Adaptive Deinterlacing 


Estimate motion at each pixel 

Use Motion value to cross-fade spatial and temporal 

interpolation at each pixel 

- Low motion means use more of temporal interpolation 
High motion means use more of spatial interpolation 

Quality of motion detection is the differentiator 

Motion window size 

- Vertical detail 

- Noise 


Edge-Adaptive Deinterlacing 


• Moving edges are interpolated cleanly by adjusting the direction of 
interpolation at each pixel to best match the predominant local edge 


One Field One Field 

Angled Line Angled Line 



Line Doubled using 
Edge-adaptive Interpolation 



Line Doubled using 
Vertical Interpolation 


Scaling 


Linear Scaling 

Resolution conversion 

- PIP/PAP/POP 

Nonlinear scaling 

- Aspect ratio conversion 

Variable scaling 

Keystone correction 

Warping 

Resampling based on a mapping function 


Upscaling 



Interpolating 
low-pass filter 
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Downscaling 
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Practical Scaling 


Textbook scaling implies you need a very large filter when dealing with 
expanded signal 

In practice you only need a small number of filter coefficients ("taps") at 
any particular interpolation point because of all the zero values 

The interpolation points are called "phases" 

e.g., scaling by 4/3 requires 4 interpolation locations (phases) that repeat - 0, 
0.25, 0.5, 0.75 

Practical scalers use polyphase interpolation 

Pre-compute and store one set of filter coefficients for each phase 

Use DDA to step across the input space using step size = (input size / output 

size) 

• Xj = Xj.! + Step 

• Fractional portion of Xj represents the filter phase for current location 

For each location, use filter coefficients corresponding to the current 
phase and compute the interpolated value 


Upscaling Comments 


* Theoretically simpler than downscaling 

Fixed length filter can be used since there is no concern about aliasing 

* However, poor reconstruction filter can introduce jaggies and Moire 

- often mistakenly referred to as aliasing. 



Quarter zone plate upscaled using replication - shows 

jaggies Quarter zone plate upscaled using interpolation - smooth 


Upscaling Comments (cont.) 


• Moire 


Introduced by beating of high frequency content with first harmonic that is 
inadequately suppressed by the reconstruction filter. 


Original sampled image: ID sine wave grating - 
cos (2*pi*(0.45)x) 

- visible Moire 




Upscaled 2X horizonta 


y using 


inear interpo 


ation - visib 


e Moire 


Upscaled 2X horizontally using a 16-tap reconstruction filter 
- negligible Moire 


Downscaling Comments 


More difficult than upscaling 

Each new scaling factor needs cutoff frequency of reconstruction filter to be altered. 
Inverse relationship between time (space) and frequency requires filter length to grow 
proportionately to shrink factor. 

Aliasing and lost information can be very visible when a fixed low-order filter is used 




Grid downscaled using fixed 2-tap filter 


Grid downscaled using filter with dynamic taps 


Scaling for PIP/PAP/POP 




> 

Main 

- PIP 






> 


PIP 


Main 



— 


> 


PIP Mode PAP Mode POP Mode 



> 

Main 

-►TeleText 

s 




PAT Mode 


Mosaic Mode 


Linear Scaling - State of the Art 


Polyphase interpolation 

Separate H and V scaling 

Typical number of phases from 8 to 64 

Typical number of taps from 2 to 8 (H and V can be different), usually more 
than 2 (linear) 


Keep in mind the fundamental differences between graphics and video 
- Graphics is non-Nyquist 

Watch out for marketing gimmicks - Total # of effective filter taps is NOT 
#taps x #phases, it is just #taps 

Correct definition of "#Taps" is how many input samples are used to 

compute an output sample 


Nonlinear Scaling for Aspect Ratio 

Conversion 


* Aspect ratio conversion is required for going between 4:3 and 
Widescreen 

- 4:3 material on 16:9 monitor 
16:9 material on 4:3 monitor 

* Several options (shown below) 


16 x 9 Display Modes 


4x3 Display Modes 
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Nonlinear Scaling Example - 
“Panoramic” Mode 
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Non-linear 3 Zone scaling 


•The input aspect ratio is preserved in the 
middle zone of the output image while 
scaling. 

•Aspect ratio slowly changes in the tail zones 
to accommodate rest of the input picture. 


Horizontal 
nonlinear scaling 



Non-linear 3 Zone Scaling - Example 



Original 4:3 image 





Linear Scaling 16:9 



Nonlinear scaling 16:9 





Vertical Keystone Correction 



Image with 

vertical keystone correction 
and aspect ratio correction 


Projection of the image 




Edge Enhancement 


Adaptive peaking 

Extract high-pass filtered version of signal 
Apply gain 

- Add back to original 

Transient improvement 

Compute derivative of signal 

Use shaped version of derivative to sharpen the transient without 
introducing undershoot or overshoot 


Picture Enhancement and Controls 


Standard Picture Controls 

Brightness, Contrast, Saturation, Hue or Tint 

Advanced Picture Controls 
- New 6-point controls - R, G, B, Cy, Mag, Yellow 

Automatic contrast and colour enhancements 

Intelligent Colour Remapping (ICR™) produces more pleasing vivid 
images 

Locally Adaptive Contrast Enhancement (ACE™) expands the dynamic 
range of the scene to provide more detail 


Color Management 

sRGB Color space for internet 


Standard Global Picture Controls 


Typically comprises of a fully programmable [(3x3) matrix + (3x1) vector] 
Color-Space-Converter (CSC) and Look-Up-Table (LUT) 

- Can be used to do linear color space transformations, standard picture 
controls (hue, saturation, brightness, contrast) and gamma correction 


Hue Saturation 



Original 




New 6-point Control 

Separate controls for 6 chroma channels - R, G, B, Cyan, Magenta and 
Yellow 

Red 



Cyan 


Intelligent Color Remapping (ICR™) 


• Example of automatic setting to enhance specific colour regions - green grass 

Greener grass 


Intelligent Color Remapping (ICR™) 


• Example of automatic setting to enhance specific colour regions - blue sky 



Locally Adaptive Contrast Enhancement 

(ACE™) 



Original 


Contrast enhanced 



Application Examples 

Basic LCD-TV/Monitor 
Fully Featured LCD-TV/Monitor 


Fully Featured MPEG-TV 


Application Example 1: LCD TV/Monitor 

without PIP 


Inputs: 

• Standard TV 

• HDTV (480p/720p/1080i) 

Output: 

• VGA - WXGA 

• 4:3 & 16:9, Progressive 

Key Features: 

• Motion Adaptive I/P 

• Film Mode (3:2 & 2:2) 

• Noise Reduction 

• CC/V-Chip/Teletext 

• Multi-Language Ul 

• IR Remote 


Basic 

LCD-TV/Monitor 


Tuner Board 


AV-AL/ AV-AR 
HD-AL/ HD-AR 
PC-Audio 


TV- In 


AV-IN (composite) 
S-VID (s-video) 
YUV 


HD-Y/HD-Pb/HD-Pr (HD) 


VGA-ln 


Tuner 
FI 12X6 


Audio 

Decoder 

MSP3450 


Audio Amp 
TDA1517 


N 




SDRAM 





PW1230 






Reference design courtesy of Pixelworks Inc. 

















Application example 2: LCD TV/Monitor 


Inputs: 

Standard TV 

HDTV (480p/720p/1080i) 

Output: 

• VGA - WXGA 

• 4:3 & 16:9, Progressive 

Key Features: 

• Motion Adaptive I/P 

• Film Mode (3:2 & 2:2) 

• Noise Reduction 

• Multi-regional scaling 

• PIP/split screen/POP 

• CC/V-Chip/Teletext 

• Multi-Language Ul 

• IR Remote 



Fully Featured 
LCD-TV/Monitor 


HD-Y/HD-Pb/HD-Pr (HD) 
VGA-ln 


DVI Rx 
Sil 161 


V-chip / CC / 
Teletext 
SAA5264 


ADC 

AD9888 



DVI-ln 





















Application example 3: Reference Design 

for MPEG-TV 



XXX 



Thank you 



